Learning Algorithms with Applications

We consider three problems in machine learning: * concept learning in the PAC model * mobile robot environment learning * learning-based approaches to protein folding prediction In the PAC framework, we give an efficient algorithm for learning any function on k terms by general DNF. On the other hand, we show that in a well-studied restriction of the PAC model where the learner is not allowed to use a more expressive hypothesis (such as general DNF), learning most symmetric functions on k terms is NP-hard. In the area of mobile robot environment learning, we introduce the problem of piecemeal learning an unknown environment. The robot must learn a complete map of its environment, while satisfying the constraint that periodically it has to return to its starting position (for refueling, say). For environments that can be modeled as grid graphs with rectangular obstacles, we give two piecemeal learning algorithms in which the robot traverses a linear number of edges. For more general environments that can be modeled as arbitrary undirected graphs, we give a nearly linear algorithm. The final part of the thesis applies machine learning to the problem of protein structure prediction. Most approaches to predicting local 3D structures, or motifs, are tailored towards motifs that are already well-studied by biologists. We give a learning algorithm that is particularly effective in situations where large numbers of examples of the motif are not known. These are precisely the situations that pose significant difficulties for previously known methods. We have implemented our algorithm and we demonstrate its performance on the coiled coil motif. Thesis Supervisor: Ronald L. Rivest, Professor of Computer Science Thesis Supervisor: Bonnie A. Berger, Assistant Professor of Mathematics Acknowledgements I thank my advisors Ron Rivest and Bonnie Berger for their ideas, help, and time. I have enjoyed working with each of them, and am grateful to them for all that they have taught me. Portions of this thesis are joint work with Ron and Bonnie, as well as with Margrit Betke, Avrim Blum, and Baruch Awerbuch. I am grateful to all of them for allowing me to include our joint work in this thesis. Thanks to David Johnson for encouraging me to go to graduate school, Shafi Goldwasser for being on my thesis committee, Albert Meyer for much good advice, and the teachers at Indian Springs School for having a continuing impact on my life. Thanks to Peter Kim, David Wilson and Ethan Wolf for help on the biology project. Thanks to the attendees of Ron's machine learning reading group for many helpful discussions. I am grateful to the National Science Foundation, Ron Rivest, Bonnie Berger, Baruch Awerbuch, Arvind, and CBCL for funding me at various points in my graduate career. Thanks to the Scott Bloomquist, David Jones and Bruce Dale for their help around MIT. Special thanks to Be Hubbard for interesting conversation, laughter, and much help. Thanks to all the members of the MIT theory group for providing such a wonderful environment. Special thanks to Margrit Betke for being my officemate and good friend for all these years. Thanks to Javed Aslam, Ruth Bergman, Avrim Blum, Lenore Cowen, Scott Decatur, Aditi Dhagat, Bronwyn Eisenberg, Rainer Gawlick, Rosario Genarro, Lalita Jagdeesan, Joe Kilian, Dina Kravets, John Leo, Roberto Segala, Donna Slonim, Mark Smith, David Williamson, Ethan Wolf, and Yiqun Yin. Thanks to my friends from the "outside" who have kept me sane. I especially thank Cindy Argo, Lisa Barnard, Kelly Bodnar, Anupam Chander, Lara Embry, Kyung In Han, Mike Hase, Pam Lipson, Lucy Song, Barbara Van Gorder, Robert Weaver, Tracey Drake Weber, and Irene Yu. Special thanks to Alisa Dougless, who has provided hours of entertainment while I've been at graduate school. I am grateful to Trevor Jim for being the best friend I could ever imagine. Most of all, I am grateful to my family. I thank my uncles, aunts and cousins for much help during difficult times. I especially thank my parents and my brothers Rajiv and Jimmy for their unbelievable courage, and for their continuing love and support. This thesis is dedicated to them.